003 (llm fundamentals, pos embedding)

estimated time: 7:30 hrs

LLM fundamentals, Token and Positional Embeddings

• Finished Chapter 1,2 of Build a Large Language Model (From Scratch)

• Read about Mistral's tokenizer https://docs.mistral.ai/guides/tokenization/

• comperhensive introduction to the positional encoding Transformer Architecture: The Positional Encoding

• 🎥 watched

  • A Deep Dive: Embeddings, Vectors & Search Algorithms in LLM's

  • Positional embeddings in transformers EXPLAINED | Demystifying positional encodings.

  • Positional Encoding in Transformer Neural Networks Explained

Notes

most of the images in these notes are property of Build a Large Language Model (From Scratch)